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(54) Title: TECHNIQUES OF IMPERCEPTIBLY ALTERING THE SPECTRUM OF A DISPLAYED IMAGE IN A MANNER 
THAT DISCOURAGES COPYING 



(57) Abstract: Copying of visual images by electronic devices is discouraged by metamerically altering the relative weights of the 
color components forming the visual images in a manner that is not perceptible to the human eye but which is perceptible in the 
copy. In a preferred technique, the visual images are represented by more than three primary color components whose relative values 
are varied in this manner. A primary application is to discourage copying by video cameras of movies displayed on theater movie 
^ screens, by causing the copies to contain unacceptable spatial or temporal variations in colors perceived by the human eye. 
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TECHNIQUES OF IMPERCEPTIBLY ALTERING THE SPECTRUM OF A 
DISPLAYED IMAGE IN A MANNER THAT DISCOURAGES COPYING 

FIELD OF THE INVENTION 
[0001] This invention concerns the protection of visual images, principally films 
and videos, from illicit recording and distribution. Specifically, this invention relates 
to providing a visual deterrent to the recording of images from a film or video screen 
by a video recording device, and the subsequent illicit redistribution of the recording. 

BACKGROUND OF THE INVENTION 
[0002] Owners of digital programming and content are increasingly reluctant to 
transmit their products unless digital media devices incorporate technologies to 
prevent them from being copied and widely disseminated without payment to the 
copyrights holders. Because digital content can be copied quickly, easily, and without 
degradation, digital programming and content owners face an exponentially 
increasing piracy threat. Digital Rights Management (DRM) systems have been 
developed which can control access to content which is in a digital file format, but 
these systems do not address the possibility of direct recording of the played-back 
content from a screen or output device. When the content is viewed by a consumer, it 
is converted to analog format, and DRM systems can no longer prevent unauthorized 
access to the content. 

[0003] The revenue model of the film industry depends on a system of staged 
release, with films typically generating most of their revenue within the first few 
weeks of initial cinematic release. Subsequent releases in other regions and in other 
formats (DVD, broadcast pay-per-view, and others) are delayed to maximize revenue 
from the cinematic release. Thus, a costly point at which film or video content is 
pirated is during the initial theatrical release. Hand-held or tripod-mounted video 
cameras are used by individuals to record motion picture films directly from theater 
screens. The recordings of these films are then copied onto blank videocassettes and 
encoded onto CDs and DVDs for illegal distribution. These illicit copies are widely 
distributed over the Internet and through physical channels even before the film's 
international cinematic release. 
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[0004] One method of cinematic release protection -employs a visual watermark, 
or 'fingerprint' in the film to identify the source of the illicit copy. These transparent 
video watermarks may be inserted into the film image at the point of production or at 
the projection point, but the resulting content can still be recorded by a video camera 
without noticeable impairment. A watermark detector is employed to forensically 
analyze suspected pirated content for a watermark identifying the print and theater 
where the recording took place. Such a video watermark may survive the recording 
process and allow identification of the projector/theatre in which the recording took 
place, but it does not prevent the recording of the film or compression of the resulting 
video. 

[0005] Many entertainment companies use encryption and copy protection 
technologies to protect their content against unauthorized distribution. TSome delivery 
systems for film content already use some form of copy protection, including DVDs 
(which use a Content Scrambling System), video tapes and digital cinema systems, 
which employ digital encryption to protect the film from copying until it is projected 
on the screen. Although encryption and scrambling techniques can be effective in 
protecting against unauthorized copying of the digital stream or file, none of these 
systems can protect against the copying of the visual content as it is being viewed 
without encryption or scrambling. 



SUMMARY OF THE INVENTION 
[0006] The present invention addresses the problem of protecting content from 
unauthorized duplication such as recording the output of a video display or cinema 
screen. The recording device is typically a camcorder which has spectral sensitivity 
that is different than the human eye. That is, while the human eye has three types of 
color photoreceptor cone cells that respond to incident light with one particular set of 
spectral component response curves, video recording devices typically use a two- 
dimensional array of light sensing elements that respond to a somewhat different set 
of three primary spectral component response curves, usually RGB {red, green and 
blue). This characteristic is utilized in the present invention by altering the spectrum 
of the displayed image in a way that does not change the perception of the image 
colors by the human eye but which does change those colors in the image captured by 
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the video recording device. These color changes result in degrading the images 
reproduced from the video recording device. This reduces the value of the copied 
video and thus discourages such illicit copying. 

[0007] In a particular application, the visual image is displayed in more than three 
primary color spectra, such as four, that are altered across the image or in time in a 
manner that is metameric to the human eye; that is, the alterations are not perceivable 
to the human eye. The value of the movie, video or other display is thus not lessened 
since the changes to the images are not noticeable to the audience. But when the 
display is copied with a camcorder or other video recording device having different 
color response characteristics than the human eye, images reproduced from the 
resulting recording are severely degraded with annoying color variations. 

[0008] There are many ways to implement the techniques of the present invention. 
For exatople, the processing can be applied to a visual image signal at recording time, 
or as part of the encoding process of protected digital media that are then distributed 
to theatres and the like for authorized viewing. In another example, the techniques 
can be applied to the output of a film or video projector with little modification to the 
projector. 

[0009] The techniques of the present invention are most advantageously applied 
to self-luminous displays; that is, displays that generate the light that forms the image. 
Examples of such displays are projectors, such as those that project images onto a 
screen, and video monitors that generate light, such as cathode-ray-tubes and 
backlighted liquid crystal devices. These can be contrasted with displays that require 
the use of external illumination to enable viewing of the image, such as print media. 
In this type of display, the spectral characteristics of a viewed image sore dependent 
upon the spectrum of the light that illuminates it as well as the spectral reflectance of 
the image itself. 

[0010] Additional aspects, advantages and features of the present invention are 
included in the following description of exemplary examples thereof, which 
description should be taken in conjunction with the accompanying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[0011] Figure 1 schematically illustrates one embodiment of the present 
invention; 

Figure 2 shows the color response of color cone cells of the human eye; 

Figure 3 illustrates a typical color response of a video camera; 

Figure 4 shows an example spectrum produced by a four color display device 
that is used in the present invention; and 

Figures 5 and 6 are example spectral power distribution curves of an image 
that are metameric to the human eye. 

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 
[0012] The techniques of the present invention utilize principles of color 
perception. Color perception is a result of radiation in the visible spectrum exciting 
three types of photosensitive cone cells in the retina. Each type of^cone cell responds 
to incident light with a different spectral response. As a result, color perception is 
inherently trichromatic. In 1931, the Commission Internationale de UEclairage (CIE) 
defined a set of three standard color matching functions that were calculated based on 
a series of color matching experiments with human observers. The color matching 
functions are a linear combination of the average spectral response of the three types 
of cone cells. These color matching functions x {red), y (green) and z (blue), shown 
in Figure 2, specify how a spectral power distribution can be transformed to a set of 
tristimulus values XYZ that specify the complete color space for a hypothetical 
standard human observer. To calculate the XYZ values, the product of the spectral 
power distribution of an image and the color matching functions is integrated. As 
made clear by this calculation, there are many different spectral power distributions 
that result in the same XYZ values and thus the same perceived color. Different 
spectral power distributions that result in the same perceived color are referred to as 
metamers. The principle of metamerism forms the basis for all modem -color image 
reproduction. 

[0013] Similar to the human eye, a typical color video camera filters incident light 
into three spectral bands and records each band with a two dimensional array of 
photosensitive elements. The spectral sensitivity curves for a typical video ^camera 
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are shown in Figure 3. Due to numerous design constraints, the sensors in video 
cameras are not colorimetric in the sense that their spectral sensitivity is not a simple 
linear transformation of the CJE color matching functions. As a result, spectral power 
distributions that are metamers for the human eye may not be metamers for the video 
camera sensor and vice versa, and usually are not. The present invention exploits this 
difference to produce visible color distortions when using a video camera to capture a 
video or film presentation. 

[0014] While the difference in human and video camera color perception is 
significant, the resulting color distortion caused when recording with a video camera 
is generally accepted and not very annoying. This is largely because the shift in the 
recorded color from the observed color is constant in time and space. To provide a 
significant deterrent to copying a video or film presentation a method for varying the 
shift in color is required. 

[0015] Nearly all current commercial display systems make use of three primary 
color components combined either additively or subtractively to form the visual 
image. Since the human visual system is also trichromatic, a^iven perceived color is 
achieved with one unique combination of the three primaries resulting in only one 
possible spectral power distribution for a given three primary display system. To vary 
the spectral power distribution without varying the perceived color, at least one 
additional primary component is required. An example of four color components 
RGBP is given in Figure 4. The additional primary component allows for the creation 
of multiple spectral power distributions that are perceived as the same color by the 
human eye but result in different values in the video camera. By varying the spectral 
power distribution over the set of metamers temporally and spatially, significant 
visible distortions can be created in the video camera recorded copy. 

[0016] Figure 1 shows a general application of an exemplary embodiment of the 
present invention, wherein a video signal to be displayed is processed at the point of 
display. An original video signal 10 consisting of RGB values intended for display on 
a system with three primary components is processed by a color converter 20. This 
device outputs a new video signal consisting of RGBP values representing the relative 
light levels of four primary components. The selection of the RGBP output value for 
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a given input RBG value is controlled by a control signal 100. This processed signal 
is then projected on the screen by sequentially displaying the four primary 
components using a single light source 40, a rotating color wheel 50 with four 
primary color filters (RGBP), and a single spatial light modulator device 60. An 
image 70 on the screen appears visibly unimpaired to an observer but when the image 
is recorded with a typical video camera 80, a resulting image 90 is visibly impaired. 

[0017] The processing of the RGB signal to produce the RGBP signal is designed 
such that the RGBP values for a given pixel produce a spectral power distribution that 
is metameric with the original intended RGB value. One means to accomplish this is 
described below. 

[0018] Given an RGB value intended for display on a system such as mat 
specified in CCIR Recommendation 709 for HDTV, the intended color in the XYZ 
coordinate system can be calculated as follows: 
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Making use of the color matching functions for a standard observer shown in Figure 2 
and the spectral power distribution of a hypothetical 4 primary display system shown 
in Figure 4, a similar equation can be calculated for converting between RGBP and 
XYZ values: 



= xyz3xioi rgbpLoi 



where xyz is the matrix of color matching functions and rgbp is the matrix of 
primaries both specified at 4 nm intervals between 380 and 780 nm. This equation 
simplifies to: 
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Making use of singular value decomposition the equation can be specified as follows: 
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where U and V are orthogonal and S is the matrix of singular values and has die form: 
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Defining the following matrices 
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As can be seen in this equation, the value of t$ is not dependent on the value of XYZ 
and can thus be changed without affecting the resulting XYZ value. Therefore the set 
of RGBP values that will produce the same XYZ value can be defined as: 
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[0019] Figures 5 and 6 demonstrate two spectral power distributions with RGBP 
values of [03, 0.5, 0.8, 0] and [0.2616, 0.5535, 0.2846, 1.0] respectively that both 
result in the same XYZ value of [0.4285, 0.4467, 0.8348]. To calculate the RGB 
value measured by the video camera, the product of the spectral power distribution 
and the video camera's spectral sensitivity functions shown in Figure 3 is integrated. 
For the given pair of RGBP values, the resulting camera RGB values are ^0.1848, 
0.4511, 0.3516] and [0.1764, 0.5965, 0.3769] respectively. These are clearly two 
different colors as viewed by the video camera sensors. With the above results, the 
control signal 100 can vary the RGBP value that the color converter 20 outputs Tor a 
given RGB input over the set of RGBP values that produce the same XYZ response 
and thus the same perceived color on the display. 

[0020] In one embodiment the RGBP value is varied in time to produce a 
temporal color modulation in the recorded image. This may be a slow variation over 
the set of RGBP metamers resulting in a slowing varying color distortion or quick 
transition between two metamers with significantly different spectral power 
distributions resulting in a flashing or flickering in the recorded image. In another 
embodiment the RGBP value is varied spatially to produce a visible impairment of an 
arbitrary shape in the recorded image. The shape may be selected to fonn text on the 
recorded image indicating that the recorded image was copied illegally or possibly the 
time and location of the presentation. 

[0021] There are also numerous types of display systems that can be used to 
create the protected image. It is only necessary, in order to carry out the specific 
techniques being described, that the displayed image include four or more primary 
components over a space and time that is relevant for the presentation. In one 
embodiment, two different sets of three primary components can be created from 
individual light sources such as lasers and the set of primaries selected can be varied 
temporally. In another embodiment, photographic film can be modified to contain 
four or more primary components, the selection of which can be varied during the 
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film processing. This allows application of the current invention with traditional film 
projection equipment. 

[0022] Although the various aspects of the present invention have been described 
with respect to exemplary embodiments thereof, it will be understood that the present 
invention is entitled to protection within the full scope of the appended claims. 
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IT IS CLAIMED : 

1 . A method of displaying a series of visual images in a manner that 
discourages copying by a video camera having a spectral sensitivity different from the 
spectral sensitivity of the human eye, comprising altering the spectral content of the 
visual images in a manner that is metameric to the human eye but not metameric to 
the video camera, whereby the alterations to the visual images are imperceptible to the 
human eye but are perceptible in a copy of the visual display captured by the video 
camera. 

2. The method of claim 1, wherein displaying the series of visual images 
includes projecting the altered visual images onto a movie screen. 

3 . The method of claim 1 , wherein displaying the series of visual images 
includes displaying the altered visual image on an electronic display device. 

4. The method of claim 1 , wherein altering the spectral content of the 
visual images includes spatially varying the spectral content across the visual images. 

5 . The method of claim 1 , wherein altering the spectral content of the 
visual images includes temporally varying the spectral content of time sequential 
multiple frames. 

6. The method of claim 1, wherein altering the spectral content of the 
image includes both spatially varying the spectral content across the visual image and 
temporally varying the spectral content of time sequential multiple frames. 

7. The method of claim 1 , wherein altering the spectral content of the 
visual images includes altering values of more than three primary spectral 
components of the visual images. 

8. A method of electronically processing color video data of a visual 
image, comprising: 
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representing the visual image with video data of more than three primary color 
components, and 

varying the relative weights of said more than three primary color components 
in a manner that a resulting display of the combination of the video data of more than 
three primary color components is metameric to the human eye. 

9. A method of displaying a visual image, comprising: 
representing a color spectrum of the image by more than three primary color 

components, 

varying the relative values of the more than three primary color components to 
represent the color spectrum of the image in a manner that is metameric to the human 
eye, and 

displaying the visual image with the varied relative values of the more than 
three primary color components. 

10. A method of deterring the copying of a video or film presentation, 
comprising: 

displaying a visual image on the screen using a system where the displayed 
spectrum is a combination of more than three primary spectral components, 

varying the displayed spectrum for given colors such that the resulting 
spectrum is metameric to the human eye and the perceived colors are the -same, and 

recording the visual image on the screen with a recording device having 
spectral sensitivity functions that are different from those of the human -eye and not a 
linear combination thereof, thereby producing a visible color distortion in the 
recorded copy. 

1 1 . The method of claim 1 0, wherein the displayed spectrum is varied 
temporally. 

12. The method of claim 10, wherein the displayed spectrum is varied 
spatially. 
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1 3 . The method of claim 1 0, wherein the displayed spectrum is varied both 
temporally and spatially. 

14. The method of claim 10, wherein the displayed spectrum is varied to 
produce a visible distortion once recorded that indicates that the presentation was 
illegally copied. 

1 5 . The method of claim 1 0, wherein the displayed spectrum is varied to 
produce a visible distortion once recorded that indicates the time and location of the 
presentation. 

1 6. The method of claim 1 0, wherein the displayed spectrum is varied to 
produce a visible distortion once recorded that is a function of the content of the 
original video or film presentation. 

17. A system for deterring the copying of a video or film presentation 
comprising: 

one or more display devices displaying a visual image on the screen such that 
the displayed spectrum is a combination of more than three primary color 
components, and 

a processor device that varies the relative amplitudes of the primary 
components displayed for a given color such that the resulting spectrum is metameric 
to the human eye and the perceived color is the same. 

1 8. The system of claim 17, wherein the primary components are created 
from multiple light sources. 

19. The system of claim 17, wherein the primary components are created 
from a single light source. 

20. The system of claim 17, wherein the primary components are created 
sequentially. 
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2 1 . The system of claim 1 7, wherein the primary components are created 
with photographic film. 

22. The system of claim 17, wherein the primary components are created 
hy a variable light source. 

23. The system of claim 17, additionally comprising a recording device 
having spectral sensitivity functions that are different from those of the human eye 
and not a linear combination thereof, thus producing a visible distortion in the 
recorded copy. 

24. The system of claim 23, wherein the recording device is a video 
camera. 

25. The system of claim 23, wherein the recording device is a color 
scanner. 



-13- 



! 



WO 2004/040899 



1/3 



PCT/US2003/034116 



RGB 
Video * 

Control . 
Signal 



10 



L 



20 



c 



30 









Color 




SLM 


Converter 


— ► 


Controller 




Figure 1 



CtB Color Matching Functions 




660 600 
Wav^enjjth (nffi) 



Figure 2 



WO 2004/040899 

0.1 r 



2/3 

Typical Camera Senear fteeponee 



PCT/US2003/034116 



0.09- 



0.08- 




WeveJength(nm) 

Figure 3 




WmotenQth(nm) 



Figure 4 




Figure 6 



